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Abstract 


The ability to solve problems effectively is one of the hallmarks of human cognition. Yet, 
in our opinion it gets far less research focus than it rightly deserves. In this paper we outline a 
framework in which this effectivity can be studied; we identify the possible roots and scope of 
this effectivity and the cognitive processes directly involved. More particularly, we have observed 
that people can use cognitive mechanisms to drive problem solving by the same manner on which 
an optimal problem solving strategy suggested bv ISolomonofil (1198611 is based. Furthermore, we 


provide evidence for cognitive substrate hypothesis (jCassimatisl . 120061 1 which states that human 
level AI in all domains can be achieved by a relatively small set of cognitive mechanisms. The 
results presented in this paper can serve both cognitive psychology in better understanding 
of human problem solving processes, and artificial intelligence in designing more human like 
intelligent agents. 
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1 Introduction 


Problem solving is probably the most common activity of all organisms, especially of humans. 
We deal with various problems throughout our lives, from infancy to adulthood. There¬ 
fore, it is not surprising that an extensive effort has been made to understand the cogni- 
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troduced in the field of theoretical computer science, although these models/algorithms were 
not much concerned with h uman approach to problem solving but rather with the objective e f- 


fectivity of such algorithms (jSolomonofll . I198 (tI : 


Hutteu . 12002 : 


Schmidhuber 


200J;|Hutter|,|2Q05!). 


Of these (and many others) r esearch ideas and results we especially note two. First, the 
Cognitive substrate hypothesis bv ICassimatisI (|2006l i states that there is a relatively small set of 
eomputational problems sueh that once the problems of artifieial intelligence are effectively solved 
for these, then the rest of the human intelligenee ean he achieve by the relatively simple problems 
of adapting the eognitive substrate to solve other problems. Cassimatis’ preliminary guess of what 
this substrate consist of includes reaso ning about t ime, s pace, parthood, categories, causation, 
uncertainty, belief, and desire. Second, ISolomonofll (jl986|) in his general problem solving system 
applied a theorem in probability that describes the probabilistically optimal problem solving 
strategy (see Section\^ and. Theorem \2.1\ in our paper). In this paper we argue that there 
is a relatively small set of cognitive processes (mechanism s, abilities) which allow humans to 
drive the problem solving by the same manner on which the ISolomonofll ()l986l l probabilistically 
optimal problem solving strategy is based. Thus, we identify the possible roots and scope of 
the effectivity of hum an problem solvin g as well as provide an indirect evidence for Cognitive 
substrate hypothesis (jCassimatisl . [200g) by arguing that only a few mechanisms are crucial for 
the effective problem solving. 

Our resul ts an d the layout of the paper are as follows. In Seetion we describe the 
Solomonoff’ (|l986l i optimal problem solving strategy (Theorem \2.1^ . In Seetion {Proposi¬ 
tion we propose a list of six cognitive processes (mechanisms, abilities) that, we argue, 
significantly help humans to generate in the effective order (according to Theorem. \2.1l\ and in 
short time the appropriate candidate ideas for solving the (sub)problems. Thus, in our opinion 
these are the mechanisms that give rise to the effectivity of human problem solving and as such 
should be implemented in cognitive architectures for this reason. Seetion^d.ea\s with the gen¬ 
eral human problem solving mechanisms. In Seetion \4.1\ { Proposition \4-^ we propose 4 classes 
of processes related to a certain outcome in the problem solving process (e.g., new informa- 
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tion, problem manipulation), and we give arguments for the hypothesis {Hypothesis that 
all conscious actions the solver does during the problem solving fall into one of these classes. 
In Section \4-^ {Proposition \4-5^ we present a subtler version of the 4 classes of processes from 
the previous section. We give arguments for the hypothesis {Hypothesis \4-6\l that the presented 
processes (mechanisms, abilities) are sufficient for the general human problem solving. Finally, 
in Section \4^ {Proposition E3 Hypothesis \4-^ we link these general human problem solving 
mechanisms with the effective cognitive mechanisms from Section 0 In Section 0 we use the 
results of Sections 0 and 0 to draw conclusions about the scope and roots of the effectivity of 
human problem solving (Hvvothesis \5. 1\) . We relate this conclusion to some other results in the 
area of general human problem solving. 

Human problem solving is an extensively researched area, and many authors have already 
addressed related issues; we claim no primacy here. Nevertheless, we believe that we are the first 
to explicitly argue that the general human problem solving mechanisms from the Proposition 
4-5\ aie necessary and probably sufficient for general human problem solving. More importantly 
though, to our best knowledge we are the first to establish the link bet ween human problem 
solving and the effective problem solving strategy used by Solomonoff (|l98(ll l ( Theorem \2.1\ 
Corollary \2.2l Provositions \S.ll \4-5\ \4-T\ and Hypotheses \4-^ I5.jp . thus drawing important 
conclusions about the scope and roots of the effectivity of human problem solving as well as 
establishing the framework in which this effectivity can be further studied. By understanding 
the principles behind the effectivity of human problem solving, the development in AI (like 
cognitive architectures) can advance along more specific paths towards attaining human level 
abilities. 


2 The effective problem solving strategy 


Let us call by Blind search a method of solving problems by checking all potential solution 
candidates. It follows that this method can solve any problem (provided the problem has a finite 
solution), although the time required, which depends on the number of solution candidates, is 
usually huge. Therefore, Blind search is a universal but no t an effectiv e meth od, if there are 
too many solution candidates to check them one by one. ISolomonoffl (|l986l l in his general 
problem solving system exploited a theorem in probability stating that if an appropriate order 
could be imposed on the solution candidates, then checking them in this order would yield 
probabilistically optimal problem solving strategy. 


Theorem 2.1 (|Solomonoffl . I198 (tI L Let mi, m 2 , m 3 ,... be candidates/ideas that can he used to 
solve a problem. Let pk be probability that the candidate nik will solve the problem, and tk the 
time required to generate and test this candidate. Then, testing the candidates in the decreasing 
order gives the minimal expected time before a solution is found. 

Corollary 2.2. The effectivity of problem solving depends on 

1. Knowledge and experience, 

2. Ability to generate — in the effective order f Theorem \2. ill and in a short time — the 
appropriate candidate ideas for solving the (sub)problems. 
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3 Cognitive mechanisms related to the effective human problem 
solving 

In this section we propose a list of six cognitive abilities or mechanisms that, we argue, sig¬ 
nificantly help humans to generate in the effective order and in short time the appropriate 
candidate ideas for solving the (sub)problems. Thus, in our opinion these are the mechanisms 
that give rise to the effectivity of human problem solving, and as such should be implemented 
in cognitive architectures for this reason. 

Proposition 3.1. The following processes (mechanisms, abilities) are related to the effectivity of 
human problem solving (with respect to Theorem \2.1\. especially to the second point of Corollary 

roi) .- 

1. Discovering similarities 

2. Discovering relations, connections, and associations 

3. Generalization, specification 
4- Abstraction 

5. Intuition 

6. Context sensitivity and the ability to focus and forget 


3.1 Discovering similarities 

Hypothesis 3.2. Similar problems often have similar solutions. 


The argume nts for this hypothesis can be found in Case-based reasoning ([Slade 


Kolodnei 


1991 


1992 1 - a process of solving new problems based on the solutions of similar past 


problems. A general case of finding a similarity among problems is when the solver identifies 
a set of problem concepts (properties, features, structure, ...) based on which he can quickly 
access similar concepts stored in his memory that are associated with an idea how to solve a 
known problem. In such a case the new problem and the known one are similar via this set 
of concepts. According to the Hypothesis I,*?. HI this process provides the solver with relevant 
solution candidates and vice versa, the solution candidates that are not similar to the current 
problem in this way are not recalled. In terms of Theorem \2.1 \ and Corollary {2)^ the solutions 
of similar (and therefore relevant) problems with high chance of solving the problem have high 
probability value (in Theorem, \2.ff . and this is confirmed by the fact that they are recalled, 
while problems that are not similar (with probabl}0 low probability value) are not recalled. 
Huma n solvers have co g nitive abilities that support eff e etive search for similarities and 


patterns ( Beiar et ah . 1991 : Gentner et al.l . [i'99,4 iBobertson . 2003). Moreover, this ability is 


robust: the solver is able to discover partial similarities and patterns by adding, removing, or 
replacing problem concepts or parts — this enables the solver to formulate hypotheses, observe 
rules, make abstractions and so on. Given that for various reasons we often add superficial 


^Problems that are dissimilar can still be related. For example, gravitational and other physical laws can be 
used in leader election problem in distributed computing. 
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elements/concepts to the remembered problems and solutions (IMedin and Rossi . Il989lb very 
few of them are actually directly similar, and this ability is able to overcome such shortcomings. 

Thus the solver is able to (often quickly) identify and access concepts representing relevant 
information and solution candidates/ideas with high chance of solving the problem (Hypothesis 
Therefore, discovering similarities participates in cognitive realization of the second part 
of the Corollary \2.^ 


3.2 Discovering relations, connections, associations 

The role of finding related, connected, or associated concepts is similar to that of finding sim¬ 
ilarities - it provides the solver with relevant solution candidates (ideas, information) and/or 
helps him to exclude some of them from further consideration. 

Hypothesis 3.3. A rational solver does not explore solution candidates (ideas, information) 
that are not in any way related to the given problem - whatever the solver does during the 
problem solving he always has some rational reasons for doing it, however vague they may be, 
like intuition or automated procedures. 


This hypothesis follow s from the semantic network m odel of memory (iQuillianl. 


1983 


Collins and Quillian . 1969 1. spreading activation m echanism ( Collins and Loftu; 


lES 


1968 


Andersonl . 


Ohlssonl . I1992I I. and Hebb rule (jHebbl . 1 19491 1. The activation in the semantic network can 


spread only to those concepts that are connected, and because of the Hebb rule the concepts 
get connected if they were (often) held active together or in close time succession in the pasll§. 
Therefore, any ideas (which are represented by a set of concepts) how to solve the problem 
the solver gets are connected with, and therefore (at least statistically) related to, the set of 
currently or recently active concepts. 

In this way the solver effectively filters out unrelated solution candidates (ideas, information) 
represented by sets of concepts simply by not recalling them. In terms of the Theorem \2.1\ this 
is the same as assigning a low probability to solution candidates that are not likely to solve the 
problem. On the other hand, the candidates that were recalled must be somehow related to the 
current problem, and by this fact they have higher chance of leading to a solution than other, 
unrelated candidates. In accordance with Theorem \2.1\ they are also explored preferentially by 
the fact that they are recalled. 

Notice also that Hebb rule (and, temporarily, context as well) strengthens the associations 
among those concepts that the solver perceived or experienced (consciously or unconsciously) as 
related in the past. Therefore, the strongest related candidates (ideas, information) are likely 
to be recalled first. In this way the solver is able to effectively filter the related candidates 
(ideas, information) so that the most promisin g one s are recalled and examined among the first 
(see also Knowledge d eployment in Anderson (|l996l l and The declarative Memory Module from 


Anderson et al 


2nn4ll 


the activation of concepts is strong enongh, one concurrent activation may suffice as well (e.g., touching 
a hot stove). 
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Furthermore, complex problems that require a creative solution often have too many possible 
approaches (solution candidates) to try them one by one. Logic, knowledge, common sense, or 
experience as well as various strategies, methods, or heuristics can be used to deduce conditions 
that all candidates must satisfy in order to lead to a solution, thus eliminating candidates 
with no chance of success. As an example, consider Zebra puzzl^ (also known as Einstein’s 
puzzle) where there is a set of houses with several properties (e.g., colour, resident’s nationality, 
kept pet and others), and the solver is given a number of constraints based on which he is 
required to determine who owns a zebra. In this problem logical inference can be used to 
eliminate wrong solution candidates because they contradict some of the constraints. The act 
of ignoring these candidates (i.e., excluding them from further consideration) can be viewed as 
assigning these candidates a very low probability of success in accordance with Theorem \2.1[ 
thus saving the solver time by not examining them. Moreover, the same mechanisms can be 
used to generate or discover additional related s olution candidates as well. For e xamp le, a well 
known general problem solving strategy TRIZ (jAltshulled . Il994l . l2000l : iKaolanl . Il996l l exploits 
already solved related problems to suggest potential solution candidates/ideas. Fu rther genera l 
problem solv i ng stra t egies also rely on finding related problems and situations (jPolval . 119571 : 


Newell 


1981 


Woodsl . [ 2000 ). 


As was the case with finding similarities, the solver may need to invest time in the identifica¬ 
tion of these relations and associations. However, a solver that does not look for and find some 
vital information (e.g., a crucial association) may be forced to check many solution candidates 
before finding a solution, hence spending a lot of time. Additionally, an ineffective solution that 
requires lot of steps increases a chance of making an error. For example, consider Gauss’ prob¬ 
lem of summing integers between 1 and 100. The solver can start to laboriously add the numbers 
thus spending a lot of time with an additional high chance of making an arithmetic error, or he 
can look for relations that will allow him to find the solution in a much more effective way. A 
problem model that incorporates the numerical property 100-|-1 = 99-1-2 = ... = 1-|- 100 = 101 
offers a lot faster derivation of the sought result (see also beginning of Section for other view). 

Hence, discovering relations, connections and associations participates in cognitive realiza¬ 
tion of the second part of the Corollary \2.2[ 


3.3 Generalization, specification 

In case of generalization, the problem concepts are substituted with more general, more abstract 
versions. Additionally, some (specific) parts of the problem (e.g., some restrictions posed on the 
problem solution, or not important details) may be omitted. For example, consider gravitational 
force between Earth and an apple — Newton generalized this relation to the force between any 
two physical objects (by omitting the particularity of the original objects). 

More generally, assume that a new problem and a known one are similar via a set of concepts, 
and that the solution of the known problem can help the solver to solve the new one (see the 
upper part of the Section \S.R . However, the solver may not, for some reasons, be able to 

^http://en.Wikipedia.org/wiki/Zebra_Puzzle, on 28.2.2015 
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discover this similarity. It may happen, though, that the solver can discover this similarity, 
if the concepts of the new problem are substituted with more general/abstract ones because 
general and abstract concepts have generally more associations (hence the increased chance of 
discovering similarity). Thus, generalization improves human ability to find similar problems, 
and therefore improves problem solving effectivity by the same reasons as mentioned in Section 

ED 

In case of specification, it may happen that the solver has many solution candidates (ideas, 
information) to consider — too many to try them one by one and no rational reason to prefer one 
over another. Here, the additional restrictions on the problem solution and/or more concrete, 
specific concepts can eliminate some portion of these potential candidates (ideas, information) 
and reduce the problem space, hence improve the effectivity of problem solving. For example, 
it is often easier to prove a special version of a statement and then generalize it than proving 
the general case immediately. Finding a general form of the solution to the cubic equation 
{ax^ + bx‘^ + cx + d = 0) can be made a lot easier, if we first restrict ourselves to one specific 
equation (namely x^ +px-\-q = 0), and then show how to obtain this form from any other cubic 
equation (i.e., how to transform the special solution to a general one). 

Notice also that in both cases the solver may no longer solve the original problem; however, 
solutions to these general/specific versions of the original problem may help the solver to solve 
the problem because they are similar (see Section [Oil . 

3.4 Abstraction 

Hypothesis 3.4. Problems with the same gist/eore have probably the same or a similar solution. 


Abstraction is the ability to see a problem in simplified, abstract, general concepts (terms), 
i.e., construction of the problem’s gist or core which represents what is the problem really 
about (as far as solver’s knowledge, experience, and his cognitive abilities permit). For example, 
consider a problem in which two cyclists, 50 miles apart, are racing towards each other from 
opposite directions at constant speeds of ISmph and 22mph. A fly buzzes back and forth 
between the noses of the cyclists at lOOmph until it is smashed when the cyclists collide. How 
far did the fly travel before it died? This problem becomes very simple “distance = speed x 
time” problem when we abstract from the shape of the fly’s trajectory. 

Abstraction extends the diseovering similarities. Gist as such is usually smaller than the 
original problem because we omit information that is not important for the solution according 
to our problem model. Moreover, gists contain simplified and abstract concepts (again omitting 
unimportant information). As a result, working with gists reduces the problem space as many 
problems coalesce into one gist what increases the effectivity what allows superficially different 
problems to be classified as similar (i.e., recalled), if they are similar on a deeper/gist level. 
Additionally, the reduced problem space allows faster comparisons between the problems. Fur¬ 
thermore, simplified, general, or abstract concepts have more associations which also increases 
a chance for their similarity. 
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According to Hypothesis \‘J.4\ (as well as Hvvothesis VJ.iSf . problems similar on this abstract 
level are relevant solution candidates. In this way, the abstraction improves human ability 
to find similar problems, hence improves problem solving effectivity, by the same reasons as 
discovering similarities (see Section \8. . 

3.5 Intuition 

By intuition we mean something similar to a preschool child learning a grammar. The child can 
use the rules mostly correctly, but he or she cannot formulate them. However, the rules are in 
some accessible way stored in the child’s memory. 

Hence, intuition is based on the ability to find similarities and/or relations (connections, 
associations) with the only difference that now the solver cannot articulate the reasons for 
them. Thus, intuition and discovering similarities and relations (connections, associations) 
participate in cognitive realization of the second part of Corollary \2f^hy the same reasons. 


3.6 Context sensitivity and the ability to focus and forget 


Concepts in solver’s memory have, in general, many associations (jOhlssonl . Il992l i. Each such 
association is a potential clue to the problem, and hence a solution candidate. However, it may 
happen that only a few concepts are truly relevant to the solution — those that do not fit the 
context or problem situation are probably not going to be part of the solution, and therefore 
should not be considered preferentially (although they still can be considered later in problem 
solving, if nothing else worked). 

Cognitive ability that blocks these seemingly unrelated concepts helps to increase the prob¬ 
lem solving effectivity because solution candidates (represented by sets of concepts) with low 
chance of success do not get in the way of candidates with higher potential. The act of not 
recalling these probably unrelated concepts and information (i.e., solution candidates) can be 
interpreted as assigning them a low probability of success which is in partial accord with The- 
orem \2.i[ On one hand, there could be a simple, but unusual or unexpected solution hidden 
in these unrelated concepts that we can miss in this way. On the other hand, this should not 
happen very often — the gain from this mechanisms in average case outweighs the occasional 
failure. 

Similarly, the ability to focus on some (small) set of concepts and to ignore some other 
set of concepts decreases the corresponding problem space, and hence generally increases the 
effectivity of the problem solving. This ability helps the solver to better utilization of his 
limited mental resources (e.g., working memory), thus enabling him to explore more involved 
relations, associations, problem manipulations and so on. In this way, this ability supports 
hnding similarities and relations, and therefore improves effectivity of the problem solving by 
the same reasons (see Sections \3.1\ and 1,9.HD . 

Finally, there is the ability to forget, i.e., remove concepts from the solver’s working mem¬ 
ory (consciousness). The active concepts tend to return to the inactive state unless there 
is some conscious effort from the solver or there is some stimulus from the outside world 
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(jPeterson and Petersonl . ll959lP As a consequence, the amount of active concepts is reduced 
what frees some of the solver’s limited cognitive resources which possibly improves his focus 
(less concepts to focus on), and also helps him to overcome impasse or functional hxedness. 


3.7 Final remarks 

It follows that each of these cognitive mechanisms participate in making human problem solving 
effective with respect to the innate quality of these mechanisms and solver’s knowledge and 
experience database (according to the Corolla,rv \2.iS\ . However, how much effective the general 
human problem solving process really is depends on the extent to which these cognitive mech¬ 
anisms participate in this process. That is, should these cognitive mechanisms be sufficient for 
the general problem solving (i.e., should there be no other crucial mechanism of which effectivity 
we know little), the human problem solving guided by these cognitive mechanisms would be, 
according to Solomonoff, in the expected case effective. 

On the other hand, the order in which the solver tests the found candidates does not need 
to match exactly the Solomonoff order from Theorem \2.1[ This depends on the solver’s knowl¬ 
edge and experience, on the quality of his cognitive skills, and on his system for choosing 
amo ng multiple alternatives ^ — this distinguishes domain novice and ex pert problem solver s 


(see IChase and Simonl. Il973l l as well as average people from geniuses (see iDearv et al.l . 120001 1 . 
However, given these limitations the human problem solver approximates the Solomonoff opti¬ 
mal problem solving strategy, and that is, perhaps, the best he can do. 


4 General human problem solving mechanisms 

The topic of this section are mechanisms that are necessary and probably sufficient for the 
successful (see bellow) human problem solving in general case because the mechanisms described 
in the previous section, even though employed during the problem solving, may not be the whole 
story. More precisely, if we want to argue that human problem solving process is effective in 
general, then we need to establish which mechanisms are used in this process in the first place. 
Once this is done (see Propositions \4-5\ and Hypothesis we can compare these general 

problem solving mechanisms with the effective cognitive mechanisms from the Proposition \3.1\ 
(see Proposition \4. ?| and Hypothesis \4-^ - 

Note that by successful problem solving we also mean effectiv^ although this does not 
necessarily mean that the found solution is itself effective. More particularly, sometimes it 
may be easier to use a simple non-effective method because trying to discover something more 
effective may actually take more time or effort than the simple non-effective approach. For 
example, recall the Gauss summation problem from Section \3.2\ Summing the numbers is the 
most obvious and simplest of solutions, but it is also a terribly non-effective one. However, many 

"^Solver’s mood, motivation and so on may also play an important role (see Wood.4 2000), but we do not 
consid er thes e facto r s here. For implementation of emotions int o cognitive architecture (and reasons for doing 
so) see Laird ( 2008 ); Wayne and Langley ( 2011 ); Hudlicka ( 2004 ). 

®Non-effective approach being Blind search (see Section\^ . 
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solvers may be done in 10 minutes (that’s six seconds for each summation). Besides, searching 
for the relation 100 + 1 = 99 + 2 = ... = 101 may actually be a lot more difficult for them, i.e., 
much more time and effort consuming. Therefore, for these solvers the effective solution (from 
their point of view) is to use a method that is objectively non-effective (i.e., there exist much 
faster methods). Thus, we observe a trade-off between the effectivity of the problem solution 
versus the effectivity of the search method that finds this solution. Ideally, the effectivity of 
(i.e., time spent by doing) search and solution should not be very different. For example. Gauss 
(or anyone for that matter) would probably not bother to look for a sophisticated solution, if 
he was supposed to sum the numbers from 1 to 10. On the other hand, no one will try apply 
the simple summation method for numbers from 1 to 1 000 000 (at least not for a long time). 

The layout of this section is as follows. In Section \4-l\ we propose 4 classes of processes 
related to a certain outcome in the problem solving process (e.g., new information, problem 
manipulation), and we give arguments for the hypothesis that all conscious actions the solver 
does during the problem solving fall into one of these classes. In Section \4-^ we present a 
subtler version of the 4 classes of processes form the previous section. We give arguments for 
the hypothesis that the presented processes (mechanisms, abilities) are sufficient for the general 
human problem solving. Finally, in Section \4-t^ we link these general human problem solving 
mechanisms with the effective cognitive mechanisms from Proposition \3.1\ 

4.1 General human problem solving schema 

Here we present a general human problem solving schema — a set of 4 states and transitions 
among them that generalize the process of human problem solving. The purpose of these states 
is to characterize the human problem solving process in a general — not task specific — way. 
Each state represents an abstract goal the solver is trying to achieve. 

State A1 (Follow the instructions) 

The solver knows or thinks he knows what to do, and he follows this plan (i.e., executes 
some sequence of instructions). Note that this does not necessarily mean that the solver 
knows how to solve the whole problem. Take, as an example, Rubik cube. The solver 
may know how to build one side of the cube, but after this side is finished, he needs to 
find the next steps. 

Note also that the solver may generate, gather, construct new information or methods, 
and/or manipulate with the problem or situation during this execution, but in this state 
the solver does not get stuck, i.e., he always knows what to do. 

State A2 (Find a better approach) 

The solver knows or thinks he knows what he should/could do, but instead of following the 
plan, he deliberates whether this course of action (solution algorithm, problem or situation 
model, ...) is correct, best possible, or effective enough, and possibly looks for/considers 
other approaches/problem models. The solver can generate, gather, construct new infor¬ 
mation or methods related to the problem, manipulate with the problem model (perform 
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mental experiments), produce new models, or transform the problem to another (easier, 
similar, more general, ...) problem to correctly asses the situation and the effectivity of 
the current and alternative approaches. 


State B (Find any acceptable approach) 

The solver does not know what to do, and he can try to generate, gather, construct, 
observe, extract new information or methods, or he can transforms the problem to another 
(easier, similar, more general, ...) problem, or he can try to find another suitable problem 
model (be., change the current approach) so that he can make the next step towards a 
solutioqj. The difference between this and the previous state is that here the solver is 
stuck, i.e., he does not have any clear idea how to proceed/get closer to the goal. The 
solver may know of several (many) possible approaches, but he, so far, cannot assess the 
suitability of any of them. He can either try them all one by one to see if any of them 
works (e.g.. Blind search), or he can try to generate more information to exclude some/all 
of them and/or find a better approach(es) (i.e., find additional solution candidates/ideas). 


State C (Take a break) 

Incubation — the solver takes a break from the problem. Such break can help the solver 
to forget old/useless/misleading information and access new problem related information 
from long-term memory, and/or to gather new information from the environment by 
processing it in lig ht of the unsolved problem (see opportunistic-assimilation model in 


Seifert et al. 


199J). 


Proposition 4.1. At each moment during the problem solving process, the solver is in one of 
the states Al — C. 

Proof. Given a problem, the solver either knows or does not know what to do. If he knows 
what to do, he can choose to execute these steps (Al), or he can choose not to execute these 
steps. In the latter case (A2) the reasonable/rational thing to do is to think about these steps 
— to assess the suitability of these steps and/or to consider alternative approaches. If the solver 
does not know what to do, then he has no choice but to gather more information related to the 
problem within the currently chosen approach and/or reappraise his approach (B). Finally, the 
solver can (temporarily) abandon the problem at any time (C). □ 

Depending on the problem the solver can start in each of the states Al, A2, B. Subsequently, 
the solver can go from any state to any. For example, the solver may follow a learnt routine 
(Al) to build one side of Rubik cube, but once this side has been finished our solver gets stuck, 
and he moves to state B. The problem is always solved in the state Al. 

The previous abstract description of the problem solving states gives us some intuition as to 
what kind of general goals the solver pursues when solving the problem (follow the instructions 
in Al, find a better approach in A2, find any acceptable approach in B, take a break in C). Given 

®The solver can get into this state also by following a plan/heuristic that is formulated too generally/vaguely 
or for a similar problem instead, and he does not know how to precisely apply its steps. 
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these goals we now want to understand what processes (mechanisms, abilities) the solver can 
apply or use to solve the problem (i.e., to achieve the partial goals given by Al - C). However, 
in order to avoid enumerating all relevant processes the solver can employ (consciously or 
unconsciously), we will focus on some larger classes of these processes, call them macro processes, 
which we characterize through their outcome. We propose 4 such classes (or macro processes) 
which, we argue, are used in successful human problem solving. We further hypothesize that 
they are also sufficient. 

Proposition 4.2. The following macro processes (mechanisms, abilities) are necessary for suc¬ 
cessful human problem solving: 

1. Generation (i.e., discovering, construction, extractioi^) of new information. 

Here are the processes (mechanisms, abilities) that provide the solver with access to new 
information, methods, problems, situations, models and so on (themselves represented 
by sets of concepts) by means of recall (e.g., discovering similar, related, or associated 
concepts), construction, derivation, inference of new concepts (e.g., application of logic, 
methods), experimentation (mental or real), or by searching external sources (e.g., library, 
Internet, colleagues). The processes can he both conscious (e.g., using logical inference) 
and unconscious (e.g., recalling a similar problem, concept). 

2. Manipulation with the information, problem or situation representation. Here 
are the processes (mechanisms, abilities) that allow the solver to manipulate with the 
problem and situation representation (model), perform mental experiments, imagine a 
different situation and so on. The solver can, for example, add new concepts to the 
problem representation, assume new problem properties/features, transform the problem 
into a different one, require additional conditions on the solution, formulate sub-problems 
and so on. In other words, the solver can change his perception of the problem and its 
situation/context (consciously or unconsciously). Note that there can be same processes 
in this class and the previous one. For example, the solver can apply logic to either derive 
new information as well as to adjust problem representation according to logical rules. 

3. Assessment of the approach. Here are the processes (mechanisms, abilities) that al¬ 
low the solver to ’’step outside” the process of solving the problem and/or to appraise 
his current or some alternative approach (including planning, monitoring, guiding, deci¬ 
sion making, comparing the situation with problem goal). Again, this class shares many 
processes with both previous classes as both generation of new information and problem 
manipulation can be used in such assessment. Note that regarding the problem solving 
effectivity the crucial part of this (macro)process is the solver’s ability to stop executing 
his current plan of action and start thinking about this plan. This may save the solver a 
lot of time by abandoning wrong/non-effeetive approaehes as soon as possible. 

4 . Incubation. The solver can always choose to give the problem solving a rest. There 
are some unconscious processes (e.g., relaxation of temporarily strengthened links in the 

^From the external world through experiment or database query, for example. 
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semantic network of the solver’s memory) that c an help the problem solving (see also the 
opportunistic-assimilation model of incubation in lSeifert et al\.\l99A) . 


Proof. Generation of new information is downright obvious. The importance of problem repre 
sentation manipul a tion comes from, for example, r esearch on i nsigh t problems (|Dunckeii . 


Gick and Holvoakl . 119801 : iKanlan and Simon . 


1945 


1990j). Maier’s (j 193 il l Two-string problem is an 


example that, without the change of initial problem representation, many test participants were 
not able to solve the problem. On the other hand, the problem with the two cyclists and a fly 
mentioned in the Section \3.4\ can be solved much faster (i.e., effectively), if the initial model of 
zigzagging fly is changed to “distance = speed x time” problem. 

The importance of the approach assessment is harder to argue but a good example is the 
solution to the more difficult version of the already mentioned Gauss’ problem of summing 
integers (see Section [37^\ . this time from 1 to, say, 1 000 000. A long and laborious approach of 
blindly adding the numbers is ineffective (in fact probably impossible), and it will most likely 
lead to arithmetic errors that will prevent the solver from ever reaching the correct solution. 
Thus, the solver needs to assess this approach as very poor, and look for a better one. Some 
arguments for this class also come from the insight problems where the solver, in order to solve 
the problem, must come to a point where he questions his (initially wro ng) approach. F inally, 
the various accounts of the importance of incubation can be found in iKoestlerl (jl96J) or in 


Dorfman et al 


(|l99fil l. 


□ 


Given the mentioned problem solving states Al - G and their associated sub-goals (or 
sub-problems), are there any important/relevant macro processes (mechanisms, abilities) not 
included in the Proposition |.^.g| that the solver uses to solve problems (i.e., to achieve the sub¬ 
goals from the problem solving states Al - G)? Perhaps, but any such process would leave 
the solver with no new information (because otherwise it would belong to the generation of 
new information group), nor it would change his perception of and approach to the problem 
and situation (problem manipulation or problem assessment group). Similarly, this new process 
cannot make the solver to stop solving the problem (incubation group). For this reasons we 
formulate the following hypothesis. 


Hypothesis 4.3. The (conscious) actions of the solver during the problem solving process can 
be classified into 4 categories form the Proposition 14-31 


On the other hand, there may be other processes (mechanisms, abilities) not directly linked 
with problem solving which, however, also play an important, albeit implicit, role in human 
problem solving (such processes can be implicitly used by the processes from the classes de¬ 
scribed in the Proposition \4-^ - For example, consider the importance of (inner) speech and 
language comprehension. It can help the solver to formulate new (sub)problems, and many 
problem solving methods are stored/remembered as mental texts (e.g., a recipe). It was shown 
that the individuals (undergraduate students) most successful in the task (Raven progressive 
matrices) necessarily used inner speech that someti mes addressed the prob l em, s ometimes them¬ 
selves, in a relatively evenly distributed manner (jPaSilveira and Gomesl . l2012l b Additionally, 
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Baldo et al. 


(j2005l ) writes that ^Hanguage being a symbolic representation system allows us to 
not simply represent concepts, but more importantly for problem solving, facilitates our ability 
to manipulate those concepts and generate novel solutions. When language is disrupted, either 
by neurological insult or verbal interference, we are left with a less sophistic ated, less flexible 


Croft and Cruse 


capac ity for analysing and solving complex problems.'^ On the other hand, 

(|200J) hypothesise that ^''language may not be an autonomous cognitive faculty in the sense that 
the representation of linguistic knowledge is essentially the same as the representation of other 
conceptual structures, and that the processes in which that knowledge is used are not fundamen¬ 
tally different from cognitive abilities that human beings use outside the domain of knowledge.’’’’ 
Either way, we believe that language certainly has a role to play in human problem solving as it 
probably is the only way in which we can convey, understand, or manipulate with complex and 
abstract concept^. Whether this is its only role in problem solving is still a matter of debate. 

Another elemental cognitive processes implicit in human problem solving are the processes 
related to the operation of working memory (support processes for executing, guiding, and 
monitoring the execution of the intended activities). These processes are essential in problem 
solving, but they cannot do any complex step in the problem solving process on their own. 
Furthermore, the solver also has the ability to interpret/understand a recalled content of his 
memory or experience. If he recalls, say, a method, then he understands which actions he ought 
to take, although it may happen that he does not know how (i.e., he knows what to do, but does 
not know how to do it). This may happen when the recalled method was originally used with 
different but similar problem. Consequently, we formulate the main hypothesis of this section. 


Hypothesis 4.4. The macro processes (mechanisms, abilities) described in Proposition^^ are 
sufficient for successful human problem solving, although successful problem solving probably 
implicitly depends on other (elemental) cognitive processes as well, including 

(a) language processing, 

(b) working memory processes (coordinating, monitoring, and executing the intended activi¬ 
ties), 

(c) ability to interpret/understand memories and experience. 


4.2 General human problem solving mechanisms 

In the Section \4Pl \ we identified 4 macro processes that are necessary (see Proposition^^ and 
probably sufficient (see Hypothesis \4.4\l for the successful human problem solving. In this section 
we are mainly interested in human approach to acquiring new information (new with respect 
to solver’s attention). That is, we replace the first macro process from Proposition 14 . with 
other, more subtle cognitive processes; other macro processes from this proposition are kept 
untouched. 

^Groups, fields, block designs are abstract mathematical concepts that probably cannot be expressed without 
proper language. 
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Proposition 4.5. The following processes (mechanisms, abilities) are necessary for successful 
human problem solving 

1. Discovering similar concepts (representing information, experience, properties, prob¬ 
lems, situations, models, ...) 

2. Discovering related, connected, and associated eoneepts (representing informa¬ 
tion, experience, properties, problems, situations, models, ...) 

3. Manipulation with the information, problem, or situation or its representation 
(e.g., transform, split, add or remove concepts, features, attributes, imagine, experiment, 

...) 

4- Assessing the situation/progress and deciding what to do next 
5. Incubation (stop solving the problem) 

Furthermore, other cognitive processes not directly linked with problem solving, including 

(a) language processing, 

(b) working memory processes (coordinating, monitoring, and executing the intended activi¬ 
ties), 

(c) ability to interpret/understand memories and experience, 
are (most likely) used as well. 


Note that by Manipulation with the information, problem, or situation or its representa¬ 
tion and Assessing the situation/progress we mean the application of some method, procedure, 
heuristic, experience, common sense, logic, ... that performs manipulation/assessment action 
on something. Also note that when we talk about human problem solving, the processes 1 &: 
2 from Proposition \4-5\ actually mean the processes 1-6 from Proposition \3.1\ (we argued in 
Section 0 that processes 3-6 from Proposition \3.1\ are special cases (or modifications) of the 
processes 1 &: 2 from the same proposition). Moreover, the items 3 &: 4 from Proposition \4.5\ 
can again be viewed as macro processes (see text above Proposition \4-2^ , rather than some 
(elemental) cognitive processes (e.g., biologically motivated). On the other hand, items 1 &: 2 
from the same proposition are biologically motivated (semantic network models, Hebb rule). 


Proof. The arguments for the processes (mechanisms, abilities) numbered 3-5 can be found in 
the proof of the Proposition 14.21 As for the discovering similar or related/associated concepts, 
they are crucial f or analogising and analogical problem solving of which immense importance 


RobertsonI (120031 1 writes: “The importance that has been attached to the role of analogising 
in human thinking cannot he emphasised too strongly. ... Andersons ACT-R and the various 
analogical models produced by Hofstadter and the Fluid Analogies Research Group, for example, 
consider analogising to be fundamental to human thinking and learning.^’’ On the other hand, 
processes (a) - (c) in the Proposition \4-^ are necessary for normal cognitive funct ioning as such, 
which also include problem solving (see also iKvllonenl . 1 19961 : 
these processes in problem solving). 


Baldo et al 


20051 for the role of 
□ 
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Let us now consider the human approach to acquiring new information (new with respect 
to the solver’s attention) - the first macro process from Proposition \^.S\ We want to argue that 
this macro process can be replaced with a combination of processes listed in Proposition |4.5| 
(all of them). We can split the processes related to the generation of new information into two 
classes: unconscious and conscious. 

Consider the first case. It was already mentioned in Section \3.S\ {Hypothesis |g.3l) that hu¬ 
mans are rational solvers in the sense that whatever we do during the problem solving we always 
have our rational reasons for doing it, however vague they may be (e.g.. intuition). This conclu¬ 
sion follows from the semantic network m odel of memory (Quillian . 11968 : Collins and Quillian . 


1969 1. spreading acti vation mecha nisms (ICollins and Loftusl. Il975l : 


Anderson. 


1983; 


Ohlsson ■ 


19921 1 ■ and Hehb rule (IHebbl . 1 19491 1. The activation in the semantic network can spread only to 


those concepts that are connected, and because of the Hebb rule the concepts get connected if 


they were (often) held active together or in close time succession in the paslfl Thus, in this 
model of memory, no information (i.e., no concept) can be accessed without it being similar to, 
related to, or associated with the currently or recently active concepts (otherwise there would be 
no path for the activation to spread there). From this we conclude that the unconscious informa¬ 
tion gathering from the solver’s long-term memory is achieved through similarity or spreading 
activation (from active concepts to related/associated concepts), and that processes/abilities 
1 &: 2 from the Proposition \4-^ play a key role here. Additionally, solver can also spot (new) 
similarities or relations/associations between concepts held in his working memory (or on a 
piece of paper for that matter). 

In case of conscious processes, the solver can acquire new information from his memory by 
recal0, or he can construct it from the information he so far gathered, or he can extract it 
from the environment through interaction, experimenting, or observation. Now, however the 
solver consciously generates the new information, it is alwavJ^ according to something, be it a 
method, heuristic, experience, or just an intuition. In all of these cases the motivation for his 
conduct (this method, heuristic, and so on) lies either in his memory (working or long term), 
or in the external world, and it needs to be fetched (by similarity or spreading activation), 
constructed, or extracted (from a database through a query, or from an environment through 
interaction, experiment, or observation) However, the construction and extraction cycle back 
because the method, according to which the solver wants to construct or extract something 
(e.g., other method), needs to be acquired prior any use. This can happen through recall 
(processes/abilities 1 &: 2 from the Proposition \4-^ together with (a) - (c)), or recursively 
through construction/extraction of another method (which again needs to be acquired etc.). 
Once a method, heuristic, experience, etc. is recalled, constructed, or extracted, the solver can 
try to apply/execute its steps. 


the activation of concepts is strong enongh, one concurrent activation may suffice as well (e.g., touching 
a hot stove). 

^°He can use strategies, methods, heuristics to boost his unconscious recall processes (see also SecMons \S.fA 

[T71fT3l) . 

^^The information cannot, of course, magically appear in the solvers head (apart from the unconscious processes 
of recall which we described in the previous paragraph). 
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Additionally, from the broader perspective, we can view the latter case as a process of 
problem solving^, and the applied/executed methods can manipulate with the problem, infor¬ 
mation, etc. as well as concern the assessment of the current approach to the problem (processes 
3 ~ 4 from Proposition \4-^ ■ 

Thus, the acquiring of new information (new with respect to solver’s attention) is in human 
problem solving carried out (mainly) by processes 1 & 2 (and also by 3 & 4) from the Proposition 
\4-^ together with (a) - (c), and this happens either directly (recall from solver’s memory), or 
indirectly (through other priorly recalled, constructed, or extracted methods)@. Consequently, 
we formulate the main hypothesis of this section. 


Hypothesis 4.6. If Hypothesis [7^ holds, then the processes (mechanisms, abilities) from the 
Proposition |.^.5| are sufficient for successful human problem solving. 


Additionally, if the solver does not know how to apply a particular process from the Propo¬ 
sition \4-^ (e.g., he does not know how to obtain new information, or how to transform the 
problem to make it easier, etc.), he can formulate this as a new problem to be solved. In this 
way, the solver creates new problems like “How do I find similar/related problems^’’ or “Is this 
a good approach? How do I assess it?” , and he solves these problems using the same process 
from the Proposition Once, for example, a similar problem is identified, the solver can 
continue solving the original problem by using the solution of this previously solved similar 
problem. From this point of view, human problem solving is a recursive process (compare this 
with subgoaling from many cognitive architectures like SOAR, ICARUS, or ACT-R). 

Notice also that the processes from the Pr opositi o n U.5| cover very well the steps/advices 
from the celebrated problem solving strategy of IPolval (jl957l i. Most ideas Polya suggests to the 
solver are related to the similar or related solved problems/information, manipulation with the 
problem, its representation, or situation, assuming new problem properties, using the environ¬ 
ment for drawing figures, and so on. 


4.3 The relation of Proposition \8.1\ and Proposition \4-5\ 

What is the relation between the general problem solving processes (mechanisms, abilities) from 
Proposition \4.^ and the effective cognitive processes (mechanisms, abilities) from Proposition 
\S.1V In this section we link these two sets of processes, and from this link we draw conclusions 
about the effectivity of human problem solving. 

Proposition 4.7. In terms of human problem solving capabilities, the processes (mechanisms, 
abilities) from Provosition \H. 1\ together with (a) - (c) from Proposition \4-5\ suffice to replace the 
processes (mechanisms, abilities) from Proposition 

Proof. Discovering similar and related/associated concepts (representing information, experi¬ 
ence, common sense, properties, methods, problems, situations, models and so on) figure in 

^^Generating relevant information can be hard, and it can become a new sub-problem to be solved. 

^^This can also be viewed as a problem solving process: the solver can formulate new problems to solve like 
‘How do I find new relevant information?" and apply the same mechanisms from Proposition\4.5\ 
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both lists of Provositions \3.1\ aiid \4.5\ Remember here the note (just below the Proposition^^ 
that the processes 1 & 2 from Proposition 1^.51 actually mean the processes 1-6 from Propo¬ 
sition \S.1\ Let us consider the manipulation with the information (or problem and situation 
representation). The solver manipulates with the information/problem according to something, 
be it a method, heuristic, experience, or just an intuition. In all cases the motivation for his 
conduct (this method, heuristic and so on) lies either in his memory (working or long term), or 
in the external world, and as such needs to be fetched (by similarity or spreading activation), 
constructed, or extracted (from a database through a query, or from an environment through 
interaction, experiment, or observation). Here we cycle back because the method according to 
which we want to construct or extract something (e.g., other method) needs to be acquired 
prior any use, and so we can apply the same processes. Once a suitable procedure is accessed, 
constructed, or extracted, the solver can try to follow its instructions. 

In this way, the manipulation with information is not some elemental cognitive process; 
rather it is a macro process that consists of two phases: acquisition of the method/experience 
according to which this manipulation is to be carried out, and application/execution of the steps 
of this method/experience. The first phase may recursively use the processes from Proposition 
but the initial incentive (i.e., the first step) always comes from the solver’s head where 
discovering similar and/or related/associated concepts (through spreading activation) is/are 
most important. The second phase comes from the solver’s ability to understand/interpret his 
own memories and experience. The assessment of the situation or problem solving progress 
follows similarly. □ 


Finally, we formulate the main hypothesis of this section - the general human problem 
solving mechanisms. 


Hypothesis 4.8. If Hypotheses Um and [/76| hold, then the processes (mechanisms, abilities) 
from Propositiontogether with the cognitive processes for 


(a) language processing, 

(b) working memory process (coordinating, monitoring, and executing the intended activities), 

(c) ability to interpret/understand memories and experience. 


are sufficient for successful human problem solving. Together, they are the mechanisms for 
general human problem solving. 


5 The effectivity of human problem solving 

So what do Sections 0 and tell us about the effectivity of human problem solving? In Section 
0 we argued that the processes (mechanisms, abilities) from Proposition \ ‘j.l\ give the human 
solver the ability to (approximately) follow the Solomonoff optimal problem solving strategy 
from Theorem, \2.1\ and in Section^we hypothesised that these processes (together with some 
additional elemental cognitive processes, see Hypothesis \4.^ are sufficient for the successful 
general human problem solving. From this it follows that there is good evidence that human 
problem solving is organized according to an optimal (i.e., effective) problem solving strategy. 
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Hypothesis 5.1. In the probabilistic sense of Theorem \2.1[ human problem solving is effective 
with respect to 

1 . solver’s knowledge and experience, 

2. quality of his processes, mechanisms, or abilities from Proposition \S.1[ 


Given what we observe in the world, the human problem solving process is indeed fast (i.e., 
effective). Whence this effectivity comes from is still uncertain, but the results of our work 
summarized in Huoothesis \ 5. l\ sussests that the roots of the effectivity of human problem 
solving lie in 


1. optimal problem solving strategy from [Solomonofig (|l98m . 

2. solver’s knowledge and experience, 

3. quality of the solver’s mechanism s from Propositi on \3. 11 

4. language processing (in the sense of 


Baldo et al 


2005|). 


Also, the human problem solving approach can be viewed as a middle path between two 
extreme theoretical problem solving strategies. The first strategy relies on database of solutions 
to all problems. This strategy solves problems quickly by a simple look-up, but the time and 
costs of building (and maintaining) such database are enormous. The second strategy, on the 
other hand, assumes no such information, but relies only on a few chosen axioms from which 
it tries to derive all solutions. While it is easy to implement such strategy, using it would 
require an enormous amounts of time. Human problem solving fits somewhere in the middle. 
We remember many problems we solved in the past of which solutions we can easily apply, if we 
were to solve them again (thus reaping the benefits of the first strategj0). On the other hand, 
we can see similarities and relations between various problems or their parts what allows us to 
apply a solution of one problem to a similar or related one. In other words, we can manipulate 
with the information we have to derive new solutions just like in the second strategy with 
the huge difference that we do not start empty handed, that is, we have more than a few 
axioms to start with. This greatly increases the range of problems we are able to solve in a 
reasonable time, while keeping the database of problems/information manageable (compared to 
the database from the first strategy). 

Additionally, whe n we compar e the h uman problem solving approach with the general prob¬ 
lem solving system of ISolomonoflg (|l986l b both based on the optimal problem solving strategy 
from Theorem \2.1[ we can see that they differ in how they estimate the values of pk for the 
potential solution candidates. While Solomonoff is using the algorithmic probability (more pre¬ 
cisely, its approximation), human approach is based on the similarity and relations/associations 
between the current problem and the solvers knowledei^. It is our belief that these two pro¬ 
cesses, mechanisms, or abilities (together with some others, see Hypothesis \4-^ are crucial for 
also bearing the costs as it takes us years to become expert problem solvers in a particular domain 


( Ericssonl . [200J). 

^®We note that finding similarity or relation between objects can be very hard. For example, su bgraph 


isom orphism problem, which can be considered as way of similarity between two graphs, is NP-complete (ICook 

Eizil). 
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the effectivity of human problem solving (s ee Hvvothesis \5.1\i . On the other hand, it seems that 
they are not used in the ISolomonofll (jl986l l general problem solving system, and also, it is not 
clear how effectively the algorithmic probability or its approximation can be computed in the 
general case. Either way, understanding the principles behind the effectivity of human problem 
solving is a sure way of achieving the human-like artificial intelligence because we already know 
that they work in humans very well. 

Finally, the fact that the effective cognitive mechanisms from the Proposition \S.1\ (together 
with a few additional elemental cognitive processes, see HyyothesisH.S^i are sufficient for human 
general problem solving gives a strong argument for the ICassimatisI (j2006l l Cognitive substrate 
hypothesis (see Section\^. The fact that his proposed mechanisms for the hypothetical cognitive 
substrate (i.e., reasoning about time, space, parthood, categories, causation, uncertainty, belief, 
and desire) do not match the mechanisms from the Proposition \S.1\ may not be a problem 
because our theory is concerned with the human general problem solving process only, while 
Cassimatis’ aim is to capture the whole, domain-universal human intelligence. 
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